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OTmTS ON PROFESSOR i\LKIN‘S :AP£R BiTIlI.ED 
'*B^MATIKG 'IHE COST-EFFECTWENESS OF 
LNSimOIONAJ PROGRAMS" 

Marvin Hoffenberg 

The "games discussants play" are many, and I have chosen the 
"savants game," the presumptuous one of donning the scholar *s 
goi^n. I prefer this "game" for this Conference since I think it 
important to relate Professor Aikin*s paper to the diverse strains 
of intellectual thought that underlie the acceptance of methodolo- 
gies to assist in choosing betiveen alternative actions. Professor 
Alkin*s paper represents one attonpt— through cost-effectiveness— 
to apply such methodologies to decision-making in education. 

In reading Professor Aikin*s paper, one can discern three 
recent trends in the applied behavioral and social sciences. First, 
there is an increasing effort toward rationalization— in the econo- 
mist's meaning of the term- -in the decision process in both the 
public and private sectors, (hrer the past two decades various 
approaches and techniques have been developed as aids (crutches 
may be a more appropriate term.) for a decision-maker to rationalize 
his decision process. Inclndf^d here are the programirdng techniques, 
extensions of benefit-cost analysis (often referred to as cost- 
effectiveness or cost-utility analysis), simulation, operations 
analysis, systems analysis, and program budgeting. IVhatever term 
is used, the common property of them all is that they aid in choos- 
ing between alternatives on the basis of relative merit. 



In government, \<e see the dev'eloping Federal planning-program- 
ming-budgeting system and its iii^rialistic spread to subnational 
uni.ts, as veil as tlie Department of Defense *s cost-effectiveness 
approach to the analysis of military problems. In industry, it is 
ncsv commonplace to choose corporate objectives through a s>^stematic 
search of alternatives and to implenent plans based on maximizing 
the opportunity of goal attainment, at least cost and risk. That 
coii^iex questions such as Federal budgeting, military effectiveness, 
and corporate strategy can be attacked through rational processeo 
of analysis no longer seems to be questionable. 

There is a difference bet^veen attacking a problem and solving 
a problem. Social, and I would include educational problems here, 
are rarely "solved" (depending on the definition of "solved"); more 
commonly, they are transformed into more tractable problems. The 
grand optimum is difficult, if not impossible, to handle in an inaly- 
tical manner. The problem is broken doim into sub-analysis , based 
on lower order criteria that are hopefully consistent with the higher 
order ones — a sub -optimization (Hitch, 1959) . Sub-optimization is 
greatly influenced by the hierarchical level at which choice is to 
be made. (The scope of the decision-making power may be another 
definition of sub -optimization.) Professor ALkin’s paper does not 
clarify Just iho is the decision-maker, the school administrator, 
the school board, and so on. 



This point needs further elaboration since it is an important 
one. The methodologies for assisting in rational choice require 
decision rules. For example, in dealing vn.tli a grant program the 
U- S. Office of Education may v:ell vjant to choose on the basis of 
total social costs and total social benefits for a v;ide geographic 
area. On the other hand, the local school board is likely to 
choose on the basis of minimizing local budget costs and to rank 
programs according to how little money they have to match federal 
funds. It does make a difference whose ox is being gored. 

A second theme reflected in Professor Alkin*s paper is the 
institutionalization of the search for problem areas and for al- 
ternative means to handle such problems = Decision-making is con- 
cerned only with the future and with alternatives . No decision 
about past events is necessary; they are "sunk costs," and no 
decision is necessary if there is no alternative. The "name of 
the game" in cost-effectiveness is alternatives . Such methodolo- 
gies as cost-ef fectiveness , systems analysis, etc., are no cure-all; 
at best they are organized methods for conceptualizing multi-dimen- 
sional problems and selecting more objectively among open alterna- 
tives. They are no better than their simulations, no better than 
the ingenuity of their designers idio must, after all, invent the 
alternatives to be tested. Incorrect models, unrealistic boundaries 
ill- conceived alternatives, and false objectives can lead to choices 
which may be worse than choosing at random. 



At this point \<e must recognize that in applying rational 
methodologies and in institutionalizing research in education, we 
must place these elements into an institutional framework and 
cease thinking of individuals as individuals making choices. The 
fact that idiat we are considering is, in actuality, institutional 
decision-making leads to a third current intellectual theme: the 

beginning of a general theory of organizational behavior. (I pre- 
fer the term organizational behavior to the more pejorative one, 
bureaucratic behavior.) A general tlieoiy is relevant to large, 
complex organizations in any field, both in the public and private 
sectors. I assume from casual empiricism that school systems are 
large, complex organizations. 

The problem posed here by Professor Alkin, since he explicitly 
recognizes that he is dealing with organizational behavior, is hoiir 
to combine the "rational" decision process of cost-effectiveness 
with the behavioral processes of institutional decision-making . I 
would prefer to look at this problem not as the "rational" versus 
the "nonrational" but as ti^o "rational" approaches vdth different 
decision rules. 

If all members of the organization shared the same values, 
desired the same objectives, and had conplete information, if there 
were no uncertainty, if the school system were relatively closed. 
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and so forth, the too rationalities could yield practically iden 
tical dioices. But we loiow that such necessaiy and sufficient con- 
ditions do not exist. There are cognitive limits to rationality as 
exemplified by the cost-effectiveness approach (^lardi and Simon, 

1958). Organizations have no operational goals; their members do. 

In such cases where resources are scarce and where subunits (indi- 
viduals) have different preference functions, exchange mil take 
place. Hiis situation leads to a definition of an organization as 
a coalition bargaining over side payments . A critical role for 
cost-effectiveness in such a situation is to aid in defining the 
meaningful boundaries for bargaining or adversary proceedings. 

Professor Alkin has posed for us a problem of profound com- 
plexity in his folloiving statement: 

The model maintains that evaluation cannot take place 
without considering ^ nature of the instr uction^ 
parameters in the program being evalu ated, without 
mder standing and quantifying the impact ot individual 
3Tid organization al contexts that infring e upon the pro- 
gram, ^d without considering a multiplicity of out cone 

measures . 

I infer from this statement that he is looking at education not as a 
single problem nor as a simple aggregation of problems but rather as 
an aggregation in which the elements (or sub-elements) are dynamically 
integrated because society perceives them as an entity. For example, 
a characteristic reaction to the Colemari Report was "IVhat the hell. 

Wliy give these educational people any more money? Give it to Sargent 
Shriver”--a direct recognition of the integrative nature of poverty 




and educational performance. Analytically, I would like to look at 
Professor Alkin's statement as a trinity whose sacredness should be 
left to the reader’s judgment. First, there is the specific problem 
area of concern. The stated problem area is the evaluation of in- 
structional programs; and the focal unit is, at the minimum, the 
individual school. Here, the school is viewed as a system, i.e., 
con5)osed of connected elements. The element of connectiveness should 
be of major interest. But what is the system definition for the unit, 
and idiat is connected? 

Professor Alkin correctly implies that he deals with open sys- 
tems. Open systems operate with continual inputs and outputs, with 
dynamic functions and everchanging states. Such systems may not be 
self- regulating, and findings concerning them may not be replicable. 
Direct intervention at various decision points may be necessary to 
regulate the system and keep it within tolerable bounds. Difficult 
methodological problems are raised by such a conceptualization. iVhat 
I suspect is needed is the subject matter of this conference: a 

general theory of instructional evaluation in order to unify evalua- 
tions. 

The first point leads to the second point of my trinity, the 
methodologies of evaluation. Professor Alkin has "zeroed in" on 
cost-effectiveness as an evaluation technique. IVhat I assume he 
means by that term are the various techniques developed for decision- 
making under conditions of scarcity and uncertainty. By uncertainty. 



I include both statistical uncertainty (errors of observation and 
estimation) and, idiat is more important, the uncertainty of future 
events . 

In dealing vath the evaluation of instructional programs, for 
let us say, the single school, there is the problem of identifying 
the instructional variables. The school is the "black box" receiving 
inputs and spewing out outputs which, in turn, lead to certain out- 
comes. This is a transformation process involving a production 
function: the relation of inputs of productive services per unit 

of time to outputs of products per unit of time. To the best of my 
kncwledge, there is considerable controversy over the relevant vari- 
ables- -on both the input and output sides — to be included in the pro- 
duction function for primary and secondary education. 

Qiie illustration of our difficulties was an interchange at this 
session on the question of interactions being an intervening variable. 
I was following the argument until another canment was made idiidi 
struck me from an entirely different point of view. This comment 
was that the intervexiing variable had to do with the solution of 
differential equations to measure how v/ell an individual would per- 
form as an adult in his future life experiences. As the discussion 
progressed, it became clear to me that the intervening variable was, 
as used, a proxy variable, although what it was a proxy for remained 
undefined. I suspect that much of the current uncertainty about the 
production function hinges on the distinction between a proxy and an 
intervening variable. 



Questior^ concerning proxy variables often relate to the environ- 
ment external to the school. I mil not say school •’'systej?.*' in the 
systemic meaning of tlie term since external elements may be brought 
into the "system" through the connectiveness of the educational pro- 
cess neti*:orlc. I now have an entry point to mention the tnird element 
of my trinity, the environmental matrix. Professor Alkin, in his 
model formulation, sets the school in an environmental matrix. This 
step logically follows from the openness of his school "s>’^stem'' in 
that it constantly interacts with the society arouiid it. I hasten 
to add that there is both interaction and feedback in this s>^stems 
neti\Torlc. 

There is one striking £sat>ire about the environmental matrix 
for education today- -its turbulence. The revolutionary times we live 
in have introduced increasing uncertainty into our mental projections 
of future society — uncertainty about its values, its technological 
requirements, the knoi^ledge that mil enable an individual to work 
and live adequately in it. The openness of the school system is 
profoundly affected by the turbulent system it interacts with and, 
more directly related to this conference, by the core of evaluation- - 
the criteria used. As one example, let us be aware of the conflict 
beti^een the black ccmimunity and the schools over evaluating the 
processes and outcomes of our current instructional programs. 




l*he preceding line of thinking leads, in opinion, to a 
systemic approach to tlie evaluation of instructional programs, that 
is, to idiat is loosely referred to as systens analysis. Vdiat is 
technically teimed cost-effectiveness (benefit-cost, cost utility) 
is only a part of the systems approach. The backlog of systems stu- 
dies in physical dcEiains has helped create a new attitude toward 
dealing mth complex questions about ccraplex systens and is being 
transferred to problems involving people systems. Our ejtperience 
with this transference is ambiguous, and it is apparent that much 
more needs to be learned about dealing with people systems . At 
this point in time, this need is particularly pronounced concerning 
the educational system and the evaluation of tnat process and its 
output. We still need to proceed on a lower level of abstraction 
with specific elements. However, we can and should begin to work 
on the systems problems since methodologies need to be developed 
to integrate the various elements. 

Let us remember that social facts are political, in the best 
sense of the teim, and should be so recognized in our evaluations. 

As an example, one can review the history of the ’T^foynihan Report” 
(Subcommittee on Executive Reorganization Hearings , 1966) . The very 
selection of facts and the evaluation of such facts are part of our 
value systems. They influence hoi^ i^e select, hoi*/ we evaluate, and 
how we appreciate the facts. They also mean that, especially in 
education, we are dealing x«/ith multivalued outputs. 



In the taper presented here. Professor AUcin has laid out for 
us a process for evaluating instructional prograiTiS; he has left the 
operations for further research. If w interpretation of what 
underlies his presentation is correct, then I remain sympatlietic. 

As I have intimated, there are difficult and stimulating problems 
in making his model operational. The mfoimational requirements are 
great; the criteria problems, tough setting; the objectives, diffi 
cult. But, as an outsider, I must ask vhat has been going on in 
research in education over the years. 
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